Finite Dimensional Optimization Part I: The KKT Theorem1

These notes characterize maxima and minima in terms of first derivatives. I focus primarily on maximization. The problem of minimizing a function f has the same solution (or solutions) as the problem of maximizing −f , so all of the results for maximization have easy corollaries for minimization. The main result of these notes is the Karush-Kuhn-Tucker (KKT) Theorem, recorded as Theorem 3 in Section 4.4. The KKT Theorem was formulated independently, first in Karush (1939) and later in Kuhn and Tucker (1951). Karush’s contribution was unknown for many years and it is common to see the KKT Theorem referred to as Kuhn-Tucker (and I still sometimes do this in my own notes). These notes cover only necessary conditions, conditions that solutions to maximization problems must satisfy. Part II of these notes discuss how to guarantee that a candidate solution is indeed a maximum (or a minimum, or an inflection point, or saddle point, etc.). Part III of these notes develops some of the complementary machinery of convex programming. One of the main attractions of convex programming is that it extends to situations where the functions are not differentiable.